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Abstract 

The common approach of designing a communication device is to maximize a well- 
defined objective function, e.g., the channel capacity and the cut-off rate. We propose 
easy-to-implement solutions for Gaussian channels that approximate the optimal re¬ 
sults for these maximization problems. Three topics are addressed. 

First, we consider the case where the channel output is quantized, and we find the 
quantization thresholds that maximize the mutual information. The approximation 
derived from the asymptotic solution has a negligible loss on the entire range of SNR 
when 2-PAM modulation is used, and its quantization thresholds linearly depend on 
the standard deviation of noise. We also derive a simple estimator of the relative 
capacity loss due to quantization, based on the high-rate limit. 

Then we consider the integer constraint on the decoding metric, and maximize 
the mismatched channel capacity. We study the asymptotic solution of the optimal 
metric assignment and show that the same approximation we derived in the matched 
decoding case still holds for the mismatched decoder. 

Finally, we consider the demodulation problem for 8PSK bit-interleaved coded 
modulation(BICM). We derive the approximated optimal demodulation metrics that 
maximize the general cut-off rate or the mismatched capacity using max-log approxi¬ 
mation . The error rate performances of the two metrics’ assignments are compared, 
based on Reed-Solomon-Viterbi(RSV) code, and the mismatched capacity metric 
turns out to be better. The proposed approximation can be computed using an 
efficient firmware algorithm, and improves the system performance of commercial 
chips. 
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Chapter 1 


Introduction 

A common goal of designing a communication system is to minimize the chances 
of getting incorrect messages, given the transmission rate and other implementation 
constraints. However, the actual error rate non-trivially depends on the configuration 
of the entire signal chain and the channel condition; thus it cannot be used as the 
cost metric when optimizing the design of a simple block. 

Various objective functions have been proposed (e.g., the channel capacity and the 
cut-off rate) to be used as alternative metrics in the optimization problem. Numerical 
solutions were derived in many past studies based on these metrics. 

The objective of this thesis is to solve the above optimization problem while taking 
the implementation cost into account. We derive results that are simple to implement, 
yet approximate well the optimal solutions. We then compare the error rate between 
the solutions based on different metrics. We also show that the optimized solution 
derived in this thesis can be implemented on commercial chips with an improvement 
of system performance. 

1.1 Overview of Digital Communication Systems 

The basic structure of a modern digital communication system consists of two func¬ 
tions: error correction and analog-digital conversion (see Fig0). 

The physical channel that delivers the information is commonly modeled as a 
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Message 


Coded Message 


Channel Input 



Inferred Message Quantized Output Channel Output 

Figure 1-1: The basic structure of a digital communication system. 

randomized analog function, which corrupts the transmitted signal and creates error. 
Since the error correction codes operate in the digital domain, it is necessary to have 
the analog-digital conversion. 

The modulator maps strings of bits from the coded message to values in a fixed 
set of constellation points; then the demodulator quantizes the channel output back 
into discrete values. 

The non-ideal conversion between analog and digital signals lowers the system 
performance, so it is valuable to optimize these conversion blocks to achieve the best 
performance given implementation constraints. 

The transmitter path is usually fully characterized by the standard, but the same 
restriction does not apply to the receiver. Hence, we want to optimize the design of 
the receiver, and specifically the design of the demodulator. 

1.2 Background 

The optimal receiver, one that minimizes the error probability, should obey the maxi¬ 
mum likelihood (ML) decision rule. This ideal condition is not practically achievable, 
due to implementation constraints. These includes the upper limit of the quanti¬ 
zation resolution, the integer constraints on the decoding matrices, and the use of 
bit-interleaved coded modulation (BICM). 

There have been extensive studies on the optimal receiver design, given the above 
restrictions. 


14 






When the only constraint is the number of quantization regions, the mutual infor¬ 
mation is commonly used as the optimization metric. Numerical solutions of quan¬ 
tization thresholds were derived for 2-PAM Gaussian channels [23j. Since deriving 
a numerical solution in the general case is considered difficult, several sub-optimal 
approaches have also been discussed [smnumnM. 

When the other two constraints are also applied, the problem is commonly mod¬ 
eled as a mismatched decoding problem. A lower bound of the highest achievable 
rate for mismatched decoding is discussed in mum, which is later referred to as the 
generalized mutual information (GMI). The GMI bound is equal to the mismatched 
capacity in some special cases|2J (3], and a generalized version of GMI was recently 
proved to be a tight bound [27]. Therefore, the GMI is commonly used as the objective 
function in optimization problems BE]. 

The generalized cut-off rate (GCR) was introduced in [5] as the cut-off rate in the 
mismatched case. It is another commonly used objective function. Investigations of 
the GCR maximization problem under integer metrics constraint have been made in 

BGa]. 


1.3 Thesis Layout 

We study the optimal demodulator implementations and their approximations for 
Gaussian channels, under different implementation constraints. 

In chapter [2| we start with the simplest case, in which the decoder allows matched 
decoding, and we fold the optimal quantization that maximizes the mutual informa¬ 
tion. In chapter [3j we add the integer decoding metric constraint into consideration 
and generalize our results to the mismatched channel capacity. In chapter [4] we in¬ 
vestigate the optimal decoding metrics assignment for 8PSK BICM systems. Then, 
in chapter [5j we summarize our results and propose future work from this thesis. 


15 


1.4 Preliminaries 


In this thesis, we make the following assumptions: 

We consider the memoryless channels, with their inputs denoted by X and their 
outputs denoted by Y. 

For each channel, the output is linear to the input with gain g , and there is an 
additive Gaussian noise W independent of the input. In other words, 

Y = gX + W (1.1) 

Without loss of generality, we assume that the channels are always normalized so that 
the inputs and the noises have zero mean and unit variance. 
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Chapter 2 


Optimization Quantizer for 
Matched Decoding 


This chapter describes the approximation solutions of the capacity maximal quan¬ 
tization problem. We first briefly review the algorithmic approach of obtaining the 
numerical solution, then develop a novel approach of obtaining low complexity ap¬ 
proximation schemes by observing the asymptotic solutions. We also provide a simple 
estimator of the capacity loss due to quantization. 


2.1 Problem Definition 


Consider a scalar memoryless channel, with its transition density function charac¬ 
terized by fy\x- We assume that the channel input takes its value from a fixed 
constellation, with a given distribution Px- 

The quantization process takes the channel output and converts it to an integer 
from a finite set: {1,2, ...,1V} . We use Z to denote the quantized version of the 


channel output (see Fig. 2-1) 


In this case, N represents the number of possible quantized outputs, and we 
consider the upper limit of IV as a constraint in our optimization problem. 

The optimal quantization refers to schemes Z(Y) : M —> {1,2,..., iV} that max¬ 
imize the highest achievable information rates on such channels, which is to maxi- 
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Quantized 


Output 


2 3 


Figure 2-1: An example of the analog-digital conversion. 


mize the mutual information of the equivalent channel I(X ; Z), based on Shannon’s 
paper [26]: 


I(x-Z) = y Px,z{x, z) In iy , ( 2 . 1 ) 

A necessary condition of the optimal solution can be derived in the following 
way: Given an arbitrary mutated scheme Z'(Y), which only differs from the optimal 
solution Z(Y) in an extremely small interval [yo,Vo + 5], we calculate the change of 
the mutual information: 

A I{X- Z)^Y, ln(P x \z)AP x ,z (2.2) 

Since the optimal solution can never be improved, the choices of Z satisfy the 
following equation: 


Z(y) = argmax(Y] P x f(y\x) \n(P x \z)}- (2.4) 

z x 
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This formula takes an arbitrary quantization scheme and calculates a revised 
scheme with mutual information improvement. Thus the numerical solution can be 
calculated by repeatedly executing this operation. 

Although we are able to derive the optimal solution for any specific channel, 
sometimes it is also desirable that the quantization scheme can be adjusted when 
there are multiple possible channel conditions. For example, we may want to adjust 
the quantization thresholds for variable SNR Gaussian channels. 

The direct implementation of this adjustment requires an array of lookup tables, 
which is complex. More practically, approximated solutions that analytically depend 
on the channel parameters (e.g., SNR in the Gaussian channel case) are desired. 
Techniques like curve-fitting have been used in solving this problem[23]. 

In this rest of this chapter, we aim at obtaining such simple schemes based on the 
asymptotic solutions of the quantization thresholds. 


2.2 An Upper Bound on Number of Intervals That 
Map to the Same Quantized Output 


To describe a quantization scheme, a common approach is to describe all the inter¬ 
vals on the channel output space that are mapped to each quantized output. For 
each quantized output z, we call the corresponding set of intervals the quantization 
intervals of z, denoted by S(z). 


Since (2.4) does not directly rule out the possibility of an arbitrarily large number 
of disjoint intervals being mapped to the same quantization output, we provide an 
upper bound on that number, in order to simplify the implementation of numerical 
search algorithms and our discussion. 


Theorem 1. Given an optimally quantized K-PAM modulated Gaussian Channel, 
with the number of possible quantized outputs to be N, the number of disjoint intervals 
that map to any quantized output is upper bounded by (N — 1) + 1- 

The proof is contained in Appendix [Aj 
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A useful corollary immediately follows for the special case of 2-PAM modulated 
channel: 

Corollary 2. For 2-PAM Gaussian Channel, the optimal quantization always assigns 
single intervals to each quantized output. 

Hence in the rest of this chapter, when 2-PAM is used, we can simply treat the 
quantized output as a single interval on the real axis. 


2.3 Asymptotic Solutions to the Optimal Quanti¬ 
zation Threshold on 2-PAM Modulation with 
3 Quantization Outputs 


In this section, we consider 2-PAM modulated channel, with input constellation at 
±1, and channel output quantized into 3 possible values. 

We assume that the channel output is quantized into 3 symmetric regions :(—oo, —6), 
(—6,6), (6, +oo). We also assume that the channel input is distributed uniformly: 
Px(-l) = Px(l) = \. 


We now apply (2.4), and obtain the following equation: 


nt , ln(2Q(6 + g)/(Q(b + g) + Q(b - g))) 

9 [ \n(2Q(b — g)/{Q{h + g) + Q(b — g)) ' 


(2.5) 


Here the Q{x) is the Gaussian Q-function, defined as: 


r*+oo 


Q 0) = 




e 2 dk. 


( 2 . 6 ) 


2.3.1 Large SNR Limit 


We first consider the case where the SNR is large, i.e., g 1. 


In order to simplify equation (2.5), we quote a useful approximation, supported 
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1 


for x 1 


(2.7) 


by 0: 


Q(x ) 


\Hkx 


-e 2 


Based on the above approximation, we can simplify the RHS of (2.5) 


RHS = 

g —^--j-oo 


M*tlr?).&< s 


Mg|), b >s 


( 2 , 8 ) 


(2.8) shows that for any fixed non-zero value of b, the RHS increases much slower 


than the LHS of (2.5) when g is large. This indicates lim^oo b = 0. 


We use this fact and resolve (2.5), and the first significant term of b is thus: 


Ha) 


(2.9) 


2.3.2 Small SNR Limit 

Now we consider the case where the SNR is small, i.e., g —» 0 + . 

In order to solve for the optimal boundary, we treat g as a perturbation, and 
observe the Taylor series of the quantization boundary b: 

b = bo + big 2 + b2g 4 --- (2.10) 


Then we can perform Taylor expansion on (2.5) to obtain these coefficients. 
We define soft bit a for convenience: 


Q(b — g) — Q(b + g) 
Q(b — g) + Q{b + g) 


We simplify (2.5) to reduce the complexity of the expansion: 


\ (X 

cosh (bg) ln(l — a 2 ) + sinh (bg) ln(-) = 0 

1 — a 


( 2 . 1 !) 


( 2 . 12 ) 
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The first-order expansion of (2.11 )(2T2| indicate^] 


—a 2 + 2ab 0 g = 0 

< 


a = -g 


Q'(bo) 
Q(b a) 


(2.13) 


Equations (2.13) can be solved numerically, and the solution is bo « 0.6120. 

We also derive the second-order term, in appendix |Bj which can be expressed as: 


bi = 



(2.14) 


Theoretically, we can continue this process and solve for the solution up to an 
arbitrary order of significance; however, the complexity of the calculation increases 
dramatically, and the solutions become less valuable due to overfitting, which will be 
discussed later. 


2.3.3 Comparison with Numerical Results 


So far we have derived the asymptotic solutions of the optimal quantization threshold, 
summarized below: 

Large SNR case: = l -^- 

Snmll SNR case (up to 1st order): b s ~ 0.6120 

Small SNR case (up to 2nd order): b S2 ~ 0.6120(1 — ^) 

We extend these solutions to the entire range of the SNR as the approximated 
schemes. To eliminate the b < 0 case, we replace negative boundary values with 0. 
Now we compare these approximations with the numerical solution. 


We plot the quantization thresholds with respect to the SNR in Fig. 2-2 We also 
calculate the relative difference of the capacity compared with the numerical solution 


and plot the curves in Fig. 2-3 


We can see that all three approximations converge to the numerical result in their 
corresponding region, for both quantization threshold and channel capacity, as we 


l Q'(x) represents ■ 
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Relative Loss Compared with the Optimal Solution Quantization Boundar 



Figure 2-2: The quantization threshold against SNR. 



Figure 2-3: The relative loss of capacity against SNR. 
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expected. 


More importantly, the lst-order small SNR approximation provides minimal ca¬ 
pacity loss on the entire range of SNR. The intuition is that the mutual information 
at large SNR is not sensitive to the quantization threshold; so, as long as a scheme 
satisfies the small SNR approximation, and does not overfit in the large SNR region, 
we can expect the loss to be reasonably small. 

Thus we can also expect that the lst-order small SNR approximation provides 
minimal loss for arbitrary quantization constraints when using 2-PAM modulation. 

For simplicity, in the rest of the paper we call this approximation the small SNR 
approximation. 

2.4 A Generalized Approach to Calculate the Small 
SNR Approximation 

In this section, we generalize the small SNR approximation to the case that the 
number of quantization intervals can be arbitrarily large. 

Instead of solving the optimality equations, we can first derive the lst-order ap¬ 
proximation of capacity in the small SNR case, and then generate an effective cost 
function based on the approximation. 

The small SNR limit of the channel capacity has been discussed in pC2 , 22] for 
both scalar channels and vector channels. For a scalar channel with transition density 
function f(y\x), the approximation can be conveniently represented using the Fisher 
information: 



(2.15) 


where I(x) is defined as: 



(2.16) 
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For simplicity, we make the following substitution: 


LfeW = AiAk). (2.17) 

Then the Fisher information can be conveniently represented as: 

1{ x ) = E Y \x(L(y\x) 2 ). (2.18) 


We can easily generalize the equation for the quantized channel: 

Zquant{x) = ^ Z\X (^Y\Z,X (L(y\x)) 2 ). (2-19) 


Then we consider using the relative loss of the capacity R, defined below, as the 
cost function. 


R 


I(X;Z) 
/(A'; Y) 


( 2 . 20 ) 


With (2.15)(2.18)(2.19), the relative loss of the capacity in the small SNR. can be 
represented as: 


quant (9) 

Ez|x(Vary|^x(-b(?/|0))) 

MW) 


( 2 . 21 ) 

( 2 . 22 ) 


Now we look at the special case of Gaussian channels. We can easily derive that 
L(y |0) = gy. Then our cost function becomes: 


R = E z (VaT Y \z(y)) 


(2.23) 


Hence the problem is completely reduced to an ordinary MSE quantization prob¬ 
lem on a unit Gaussian variable. 

A list of solutions, represented using the quantization thresholds, are included in 
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Table 2.1 


The results agree with those given in [IB]. 


Table 2.1: The small SNR optimal quantization thresholds for matched decoding 


Quantization Regions 

Locations of Positive Quantized Boundaries 

3 

0.6120 

4 

0.9816 

5 

0.3823, 1.2443 

6 

0.6589, 1.4468 

7 

0.2803, 0.8744, 1.6107 

8 

0.5005, 1.0499, 1.7479 

9 

0.2218, 0.6812, 1.1976, 1.8655 

10 

0.4047, 0.8338, 1.3246, 1.9682 

11 

0.1837, 0.5599, 0.9656, 1.4357, 2.0592 

12 

0.3401, 0.6943, 1.0812, 1.5344, 2.1407 


Note that in the general case when the signals are not normalized, additional gain 
should be applied to these results. So the actual thresholds based on the small SNR 
approximation are proportional to the standard deviation of noise. 

Now we compare the performance of our proposed solution with those of the 
numerical solution and the two traditional MSE quantization schemes introduced in 


We plot the relative loss of capacity with respect to the SNR in Fig. 2-4 


m 


the case where the channel output is quantized into 5 possible values. We observe 
that the small SNR approximation has strictly better performance than the MSE 
quantizations. 

We also make a plot that compares the relative capacity loss of the small SNR ap¬ 
proximation and the optimal solution with different numbers of quantization regions 


(Fig. 2-5). The plot shows that our approximation has a stable performance under 
different quantization constraints. 


2.5 Asymptotic Loss of Capacity in High Rate Limit 

In this section we study the trade-off relationship between the capacity loss and the 
resolution constraint in the high rate limit, i.e., N —)■ +oo. 
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Figure 2-4: Comparison between different sub-optimal schemes. 



SNR(dEi) 


Figure 2-5: Performance of small SNR approximation under different quantization 
constraints. 
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We make the following assumptions: 1. Each quantized output maps to a single 
interval. 2. N is large enough so that, within a single interval, the conditional 

dP 

distribution Px\Y and its first order derivative ^ Y are nearly constant. 

We first calculate the capacity loss due to quantization, denoted by AC: 

AC = / V -.lyPsYln^dy. (2.24) 

J “7 Px\Y 


Given that Px\y is almost unchanged within each interval, we have Px\z ~ Px\y- 
Hence the following approximation holds: 


, Px\z ~ Px\z — P x\y (■ Px\z ~ Px\y) 2 

Px\Y Px\Y 2 Px\ Y 


(2.25) 


Then the capacity loss contributed from a single interval can be approximated as: 


A C z ^Y, P z 

X 


Var(Px| y ;Z) 

2Px\z 


(2.26) 


Again, given the linear approximation of Px\y for each single interval, the capacity 
loss can be represented as a weighted distortion function about Y: 


We slightly generalize the minimum distortion formula in the unweighted case, 
which was stated as equation (13) in [2D], and obtain the minimum weighted distor¬ 
tion: 


AO, 


1 ( [ fr 
127V 2 V \ ^ 2 /'\ r 


dPx\Y 

dy 


fdyf. 


(2.28) 


The above equation can be simplified under the case of normalized Gaussian chan¬ 


nels: 














/\r = 

— 1 ^ mm. 


241V 2 


V /yVar(X; Y)dyf 


(2.29) 


Now we can compare the asymptotic solution with the numerical result in the case 


of 2-PAM modulation. We plot the relative loss R, defined in (2.20), for both results, 


in Fig. 2-6 
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Figure 2-6: The relative capacity loss against N. 


The numerical solutions quickly converge to the asymptotic curves; thus the 
“inverse-square law” can be used as a low-cost estimation of the capacity loss. 

Finally, the approximated relative loss in the small SNR case can be easily calcu¬ 
lated and expressed: 


lim R « 

s—> o 


N' 2 g ->o £d 


V^TT 1 

~1Tn 2 


(2.30) 

(2.31) 
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We conjecture that the relative loss of capacity monotonically decreases with re¬ 


spect to SNR, based on the observation from Fig. |2-6| If the conjecture is true, then 
the above formula gives an extremely simple estimator of the relative capacity loss in 
the worst case condition. 


2.6 Summary 

We examined the optimal quantization problem for a scalar Gaussian channel. Three 
main results are derived in this chapter: 

1. We proved that the optimal quantization regions for K-PAM Gaussian channels 
consist of a finite union of intervals. Especially in the 2-PAM case, the region 
that corresponds to any quantized output is proved to be a single interval. This 
result corresponds to the single-interval assumption in the MSE quantization 
case, and simplifies the discussion of the capacity maximization quantization 
problem. 

2. We showed that a linear approximation of the optimal quantization thresholds 
can be derived based on the asymptotic solutions in the small SNR case. The 
scheme gives no more that 0.6% of relative capacity loss for 2-PAM modulated 
Gaussian channels. This quantization scheme can be implemented with low 
complexity (e.g. a linear analog gain control) when the receiver measures the 
channel condition and adapts its quantization strategy correspondingly. 

3. We derived a simple estimator of the relative capacity loss, based on the asymp¬ 
totic solution in the high rate limit. For 2-PAM modulation, the worst case loss 
is \ / Htt/2N 2 , if the conjecture that the relative loss of the capacity is maxi¬ 
mized at small SNR is true. This estimator gives a straightforward trade-off 
relationship between the system performance and the quantizer resolution. It 
also simplifies the system design by simplifying the process of picking the proper 
quantizer resolution. 
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Chapter 3 


Optimal Quantizer for Mismatched 
Decoding with 2-PAM Modulation 

This chapter generalizes the asymptotic approximations we derived in chapter|2]to the 
case of mismatched decoding. We derive the asymptotic solutions of the quantization 
thresholds, and study the trade-off between the loss of the mismatched capacity and 
the number of bits representing the decoding metrics. 


3.1 Problem Overview 

We consider a discrete memoryless channel (DMC), with input alphabet A, output 
alphabet y, and transition probabilities Py\x- 

Given a transmission of a coded message with length n, the receiver aims to 
reconstruct the transmitted string X n , which is known to be selected from a given 
codebook A4, based on the channel output Y n . 

We suppose the receiver is equipped with a set of symbol-to-symbol decoding 
metrics q(x,y), such that the receiver makes the decision by picking the string that 
maximizes the sum of the metrics. 
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n 


(3.1) 


X n (Y n ) = argmax{S2q(xi,yi)} 

X ^M ^ 

The maximum likelihood decision can be implemented by making the decoding 
metrics satisfy the following condition. 

q(x,y) = A\nf Y \x(y\x) + B(y) (3.2) 

where A is a positive constant and B(y) is an arbitrary function. 

ffowever, in many cases, the decoding metrics in a practical system can only take 
integers from a finite set. This prevents the optimal solution from being implemented. 
The corresponding maximum achievable rate is referred to as the mismatched capacity 
C'. 

In this chapter, we restrict our discussion to 2-PAM Gaussian channels and study 
the quantization problem under this scenario. 

We denote the quantized output alphabet as Z, each corresponding to a possible 
pair of decoding metrics q(x,z). For convenience, we define the relative metric q(z) 
as follows. 

q(z) = q(l,z) - q(-l,z) (3.3) 

We restrict our quantizer to be symmetric, that is 

v(Z(p)) = -q(Z(-y)) (3.4) 

We want to pick the quantization function Z(y) : y —>■ Z that maximizes the mis¬ 
matched capacity. 
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3.2 Background and Numerical Optimization 


The mismatched capacity C for a binary input discrete output channel Pz\x can be 
represented using the generalized mutual information Iqmi{Z] X) [3]. 

C' = ma xIqmi(X; Z) (3.5) 

Px 

where the generalized mutual information Igmi(X ; Z) is defined as: 

I GM i(X-, Z) = min {H(X) - H P ,(X\Z)} (3.6) 

P x\z 


S.t. 


£ p z(z)^i ,z(x,z) = p x (x) 


Z, p z( z ) p 'x,z( x > Z ) £ z )] 


(3.7) 

(3.8) 


Because it is non-trivial to maximize (3.5) directly, an equivalent expression is com¬ 
monly used for optimization problems. 


-p cx.q(x jZ') 

C' = max H(X) + V Px,z(x, z ) ln( x6 -' 

Px,fx,a >o v v ; E X 'fx' eaq(x ’ z) ' 


(3.9) 


For binary input symmetric channels, the capacity is optimized when Px is uniform 
and fx is a constant vector [6], 


e aq{x,z) 

C 'sym = log 2 + max ^ P XjZ (x, z ) ln( z) , 

— , Z—jx' c 


(3.10) 


Then, for the optimized quantization scheme, 3a > 0 such that the assignment of the 
decoding metrics satisfies 


Z(y) = argmax P Y \x(y\l)H 1 + e _ aq{z) ) + P Y \x(y\ ~ 1) ln ( x + £aq{z) ) ( 3 - n ) 
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From (3.11), given any two quantized outputs Zi,z 2 , the boundary of their quantiza¬ 
tion regions (if it exists), denoted by b , satisfies 


Py\x{b\l) l + e aq ^ l + e~ aq ^ 

Py\x(b\ - 1) n 1 + e aq ^ Z2 > n 1 + e~ aq ^ 


(3.12) 


So far, we have derived that the optimal quantization regions only depends on the 
scalar parameter a. The derivative of the maximum mismatched capacity as a func¬ 
tion of a can be easily calculated. 


dC'(a) = dC'(a, Z(-)) _ ^ P z]x (z\l)g(z) 
da da 1 + e a<? W 


(3.13) 


z 

Then the numerical solution of the optimal quantization scheme can be calculated by 
any ID peak-finding algorithm. 


3.3 Optimal Quantization for 2-PAM Gaussian Chan¬ 
nels 


We start by formally labeling the quantized outputs based on their relative metrics, 
and also labeling their quantization thresholds. 

Practically, a quantized output with relative metric of 0 gives no extra cost in 
the size of the decoder implementation, because no action is required in the decoding 
process. Hence, we assume there always exists a zero metric output in our discussion, 
and we label this quantized output as 0. 

For all the quantized outputs with a positive relative metric, we sort their metrics 
in an increasing order and label them from 1 to K (.K represents the number of such 
quantized outputs). Based on the symmetric quantizer assumption, we label their 
negative counterparts from —1 to —K. 

When a Gaussian channel is used, it is easy to verify that the relative decoding 


metric q(Z(y )) is an increasing function of the channel output y based on (3.12), 
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Quantized output Z 


-2 


-1 


-\ -HH—h 


Channel output Y 


Quantization threshould b 


—b 2 —bi —bo b 0 b 1 


Figure 3-1: An example of a mismatched quantization, 
given that the quantization is optimal. 

So the optimal quantization matches the quantized outputs to single intervals, 
and we can conveniently label the quantization thresholds using ±6 0 , ±&i, ±&a-i 


(see Fig. 3-1). 


The quantization intervals in this case can be expressed as 


1 1 -L 1 _i_ p-<xq(i) 

b t = — ln(-ln( A /ln( 

2 g y y 1 + e Q9h+l) 11 V X q- e -aq(i+l) '' 


(3.14) 


where a is the quantization parameter introduced in section 3.2 


3.4 Asymptotic solutions 


In this section, we solve for the optimal quantization thresholds when the SNR is 
small or large for arbitrary decoding metrics. The solutions are fully determined by 


(3.13) and (3.14). 


3.4.1 Large SNR Case 

In the large SNR case, g —> +oo. We expect the optimal quantization parameter a 


to grow to Too so that the thresholds derived from (3.14) provide non-zero solutions, 


which give a better capacity compared with a hard quantizer. 
In this case, the thresholds can be approximated as 


, aqi 
ii= 2^ + ° (1) 


(3.15) 
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Then, by applying dC d ^ 1 ' 1 = 0 to (3.13), we can derive the optimal quantization 
scheme for large SNR (for brevity, we put the derivation in Appendix [C|. 


« =2fe + °( ln ^) 

< 


bi 


— ML 
Mk 


+ o(l) 


3.4.2 Small SNR Case 


(3.16) 


In the small SNR case, g —> 0. We expect that the optimal quantization parameter 
a decays to 0. 

The optimal quantization thresholds can be approximated as 


bi 


a(q(i) + q(i + 1)) 


We can also expand (3.13) about g and a. 


dC'(a) 

da 


L(y\0)q(z) 
n o 



(3.17) 


(3.18) 


where function L(y\x) is defined as 


L(y\x) 


d\wf(y\x) 

dx 


(3.19) 


When a Gaussian channel is used, L(y |0) = gy, the optimal quantization param¬ 
eter satisfies the following equation. 


a _ E Y (2q(z)y) 
g E Y (q(z ) 2 ) 


(3.20) 


Essentially, the solution of the optimal quantization thresholds defined by (3.17) 


and (3.20) is independent of the channel gain g. Thus we can calculate the thresholds 
numerically. 

Interestingly, the optimal quantization is uniform, when integer decoding metrics 
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(i.e., q(i) = i ) are used. Later in this chapter, we will see that the small SNR ap¬ 
proximation gives negligible loss on the entire range of SNR, similar to the matched 
decoding quantization case. This explains the fact that uniform quantizers, which 
are implemented on most digital communication devices, do not give significant per¬ 
formance loss compared to non-uniform quantizers, even if the optimal solution is in 
general non-uniform. 


In Table 3.1, we list the optimal quantization thresholds for integer metric quan¬ 
tizers with different quantization region numbers (defined as N — 2 K + 1). For 
brevity, we include only the largest quantization threshold bx-i- 


Table 3.1: The small SNR optimal quantization thresholds for mismatched decoding 


Quantization Region Number 

3 

5 

7 

9 

11 

13 

Largest Quantization Threshold 

0.6120 

1.2645 

1.6269 

1.8683 

2.0460 

2.1846 

Quantization Regions 

15 

17 

19 

21 

23 

25 

Largest Quantization Threshold 

2.2975 

2.3922 

2.4735 

2.5445 

2.6074 

2.6638 

Quantization Regions 

27 

29 

31 

33 

35 

37 

Largest Quantization Threshold 

2.7148 

2.7614 

2.8042 

2.8437 

2.8805 

2.9148 


Remarkably, the optimal quantization thresholds for a 3-region quantizer have the 
same values as the solution we derived in the matched decoding case. This outcome 
is expected because at 3-region quantization, any symmetric decoding metrics are 
equivalent to matched metrics. 


3.5 Comparing with Numerical Solutions 


We compare the approximation schemes we derived from the asymptotic solutions to 
the numerical result, in the case of a 5-region quantizer with integer decoding metrics. 
In Fig. |3-2 we plot the quantization thresholds against the SNR, and in Fig. 3-3 


the relative loss of the mismatched capacity due to the asymptotic approximation. 

Similarly to the matched quantization case, we observe that all the approxima¬ 
tions converge to the numerical result in their corresponding region. In addition, the 
capacity loss produced from the small SNR approximation is extremely small, so that 
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Figure 3-2: The mismatched quantization thresholds against SNR. 


o 



Figure 3-3: The relative loss of mismatched capacity against SNR. 


this approximated scheme is almost strictly better than the large SNR approximation, 
even in the large SNR case. 

We also compare the relative capacity loss from the small SNR approximation 
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and the optimal solution with different quantization constraints. In Fig. 3-4, we plot 
the capacity loss due to quantization for integer metric decoding when the number of 
quantization regions equals 3, 5, or 7. We can see that the small SNR approximation 
has stable performance. 



SNR(dEi) 


Figure 3-4: Performance of small SNR approximation under different quantization 
constraints. 


3.6 Trade-off between Capacity Loss and Quanti¬ 
zation Resolution 

Finding the trade-off relationship between the capacity loss and the quantization 
resolution in the mismatched case is also a practically useful topic. Detailed analysis 
shows that the relative loss of mismatched capacity for integer metric decoding grows 
at the speed of (see Appendix Jd|. However, this high-rate approximation is not a 
good estimator of the capacity loss, since the convergence to this asymptotic solution 
requires y/\nN 1, which in practice is not true. 
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We also plot the trade-off curves for the matched and mismatched decoding case 


based on the numerical solutions (see Fig. 3-5). We can observe that the additional 
capacity loss due to the mismatched decoding is increasingly higher, which can be 
explained by the additional In N factor in the high rate approximation formula. 



Figure 3-5: The trade-off between capacity loss and quantization resolution for 
matched and mismatched decoding. 


3.7 Summary 

We studied the optimal quantization problem for mismatched decoding in this chap¬ 
ter. 

We showed that the optimal quantization thresholds using small SNR approxima¬ 
tion are still linear functions of the standard deviation of the noise, and they still give 
negligible loss on the mismatched capacity. 

We also showed that the relative loss of the mismatched capacity decays more 
slowly when we increase the resolution of the quantizer, compared with the matched 
decoding case. However, the asymptotic solution of the capacity loss no longer well 
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approximates the numerical result; thus, finding a simple form estimator for the 
capacity loss is more difficult. 
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Chapter 4 


Mismatched Decoding for 8PSK 
Modulation 


In this chapter, we explore the mismatched decoding problem in the case of 8PSK 
bit interleaved modulation. 

We first calculate the optimal decoding metrics assignment that maximizes the 
generalized mutual information or the general cutoff rate, and compare their error- 
rate performances. Then we introduce a low-complexity LLR decomposition algo¬ 
rithm for the 8PSK modulation, and show that the proposed demodulation scheme 
implemented on a commercial chip improves its performance. 

4.1 Problem Definition 

The bit interleaved coded modulation, invented by Zehavi [28], describes a scenario 
where a pair of bit interleavers are added to the signal chain of communication devices, 
so that each received symbol has to be decomposed to several bit metrics, and the 
decoder makes the decision correspondingly. 

Assigning the bit metrics to minimize the probability of getting an incorrect mes¬ 
sage for modulation types like 8PSK or 4PAM is non-trivial, because the a posteriori 
probability distributions of the bits are not independent given the received symbols. 
Alternatively, a common approach to solving this problem is to consider it as a 
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mismatched decoding scheme, and to maximize objective functions like the GMI or 
the GCR PIES! H 8 ]. 


Here, we focus on the case of 8 PSK Gaussian channel, with its bit-constellation 


assignment specified in Fig. 4-1 



Figure 4-1: 8 PSK bit-constellation assignment 


We assume that each transmitted symbol x is converted from three input bits b \, 
62 , and 63, and each received symbol y is decomposed into three bit metrics q \, ^ 2 , 
and q^. Based on the definition of BICM, the metrics q(x,y ) that are used by the 
decoder can be expressed using the three bit decoding metrics. 


3 

Q(x,y) = ^2qi(bi,y) 
2=1 
3 


= ^Qi(y) 1 {bi = 0 ) 


(4.1) 

(4.2) 


To simplify our discussion, we ignore the integer metric constraint and assume the 
bit metrics can be arbitrary real numbers. 
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4.2 Decoding Metrics that Maximize GCR or GMI 


Now we consider the GMI and the GCR of BICM channels under the uniform input 
distribution constraint, and we compare the error rate performance of the optimal bit 
metrics assignments that maximizes these objective functions. 


4.2.1 Decoding Metrics that Maximize GMI 


First we derive the optimal bit metric assignment that maximizes the generalized 
mutual information. This solution has also been discussed in [15]. 


From section 3.2 
pressed as 


we recall that the generalized mutual information can be ex- 


Igmi(X,Y) 


max 

fx,a> 0 


H(X) + J2 p xAx,y) ln( 

x,y 


f e aq(x,y) 

Ex- ' 


(4.3) 


For an 8PSK modulated channel with a symmetric input distribution, the above 
formula can be further simplified into 


e aq{x,y) 

I GM i(X, Y) = 3 log 2 + max ^ P x>Y {x, y) ln( ^ q{x , }y) . 

x,y 


(4.4) 


When bit-interleaved coded modulation is also used, by applying (4.1), the gen¬ 


eralized mutual information equals the sum of the GMI of three independent binary 
input channels. 


' GMI 


(X, Y) = max ^(log 2 + ^ P Bi ,Y(b h y) ln( 

a>0 Lz ' 


i =1 


h,y 


e aq(bi,y) 

E b ' e a ^Xv) 


)) 


(4.5) 


Hence, the optimal bit metrics that maximize the GMI are proportional to the log- 
likelihood ratios of the marginal probability distribution of each bit. 
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(4.6) 


L,z:b i =l P Y\x(y\ X ) 


(4.7) 


where A is an arbitrary constant. 

For 8PSK Gaussian Channels, the log-likelihood ratio equals 


LLRi = 



(4.8) 


(4.9) 


where we use 11 • 11 to denote the norm of a vector, and use • to denote the dot product. 


Practically, the max-log approximation In ^7 x* ~ max* ln(xj) is frequently used 
in the calculation of the marginal LLR. 


(4.10) 


LLRi ~ max x ■ y — max x ■ y 


x:bi =0 x:bi=l 


In this way, the approximated LLR can be calculated without an exponential 
function calculator. 

4.2.2 Decoding Metrics that Maximize GCR 

The general cut-off rate, derived as a cut-off rate in the mismatched decoding case, 
was defined in [23]. 



(4.11) 


X 


When Px is uniform, the formula can be simplified to 
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(4.12) 


Rccr = H(X)~ max In V P x , Y (x, y) 


x,y 


Thus, finding the bit metrics that maximizes the GCR is equivalent to solving the 
equation below. 


y' ' e ag(x',y) 

(91,«2,?3) = argmin Pxy(x, V ) V a (r, g ) — 


(4.13) 


<? V<?2,q3 


For 8PSK modulated channels, although deriving the exact solution to (4.13) 


is non-trivial, a simple approximated solution exists, when the signal-to-noise ratio 
of the channel is large enough that the max-log approximation can be applied (see 
Appendix |E| for the derivation). 


Q l = 


<?2 = 


A LLRi, 


LLRi < LLR-2 

A (LLRi - 

|^|sgn (LLR 1 )), 

LLRi > LLR'2 

ALLR 2 , 


LLRi LLR 2 

A ( LLR-2 - 

|^|sgn(LWR 2 )), 

LLRi < LLR 2 


<?3 — A LLR 3 

where A is an arbitrary positive value. 


(4.14) 

(4.15) 

(4.16) 


From (4.10), we know this approximated solution can also be calculated without 


an exponential function calculator. Hence it can also be implemented on hardware 
with reasonable cost. 


4.2.3 Error Rate Performances of the Two Demodulation 
Schemes 

We compare the error rate performances of the two bit-metric assignments on a 
powerline communication system developed by Maxim Integrated. We simulate the 
operation of the chip in a model written in C, and we use time domain AWGN 
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channels to evaluate its performance. 


The system uses RSV code for error correction, so we plot the bit error rate at the 


Viterbi decoder’s output, as well as the message error probability, against different 


time domain SNR (see Fig. 4-2). 


BER at Viterbi Output 



Frame Error Rate 



Figure 4-2: The error rates for two different bit-metric assignments. 


From the plots, we can see that the bit-metric assignment that maximizes the 
GMI gives a better performance, whether the Reed-Solomon code is active or only 
the convolutional code is used. We also conclude that, although the general cut-off 
rate is a function that evaluates the performance of convolutional code based sys¬ 
tems, choosing a demodulation scheme that maximizes the GCR does not necessarily 
minimize the error probability. 
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4.3 Error Rate Performance of the GMI Maxi¬ 
mization Demodulation Based on a Fast LLR 
Decomposition Algorithm 


The following text discusses the implementation of the GMI-based 8PSK demodula¬ 
tion algorithm, and the test result of its performance on a physical chip. 


4.3.1 Firmware Implementations of 8PSK Demodulation Al¬ 
gorithms 

The main task of calculating the optimal bit metrics that maximizes the GMI is to 
calculate the LLR for each bit. Several low-complexity approximations and their 
implementations have been discussed in [4( :9j I2Tj . For 8PSK modulation, we propose 
a simple algorithm that calculates the max-log approximated LLRs as defined in 

(P9- 


Proposed Demodulation Algorithm 


From (4.10), we know that in order to calculate the LLR for each bit, we need to first 


find the closest constellation points to the channel output Y that correspond to bit 
values 0 and 1, and then calculate the difference of their dot products to y. 


We divide the complex plane into 8 regions (see Fig. 4-3) so that, for each region, 


the selection of the closest constellation points remains the same. Thus the LLR 
calculation in each region can be directly implemented by calculating the dot product 
of Y and a fixed vector. 

For each channel output y, we denote the closest constellation point that satisfies 
b = 0 by x 0 . Similarly we denote the counterpart by X\. Then the approximated LLR 
equals 


LLRi « (x 0 -x 1 )-y 


(4.17) 
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Q 




Figure 4-3: The 8-region devision of the channel output space for 8PSK demodulation. 


There are effectively 8 possible values of (x 0 — Xi) (see Fig. 4-4), thus the dot 
product can be implemented with no more than 4 fixed complex multiplications. 


0 



Figure 4-4: 8 possible values of (xq — x\) 


At the same time, these dot products also provide information about which region 
contains the channel output y. Thus, no extra complex multiplication is needed to 
look up the regions. 

Now we propose a firmware algorithm that uses jump look-up to handle the LLR 
calculations in all 8 regions. 


Step 1: We calculate all 4 dot products marked in Fig. 4-5 using 2 complex mul- 
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tiplications. Because all those vectors are parallel to the boundaries of the 8 regions, 
we can also identify the region that contains channel output from the multiplication 
results. 


Q 



Figure 4-5: 4 dot products (marked as red) required to be calculated in step 1. 

Step 2: For each region, in order to calculate the LLRs for all 3 bits, only one 
extra dot product operation is needed. Thus we use another complex multiplication 
to complete the calculation. 


4.3.2 Error Rate Performance of the Proposed Algorithm 

We implemented the proposed algorithm on the powerline communication chip pro¬ 
vided by Maxim Integrated, and tested the performance of different demodulation 


algorithms using the configuration described in Fig. 4-6 A function generator is 
used to generate the noise, and we control the SNR by changing the noise amplitude. 


We plot the error probability under different noise amplitudes, for three different 


demodulation algorithms (in Fig. 4-7). 


What we refer to as the old soft phase demodulation is the original demodulation 
scheme that was implemented on the powerline communication chip, ft calculates 
the bit metrics based only on the phase of the received symbol. A direct replacement 
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Transmitter 


Receiver 


Figure 4-6: Board test configuration 

of the look-up table, which is referred to as the new soft phase demodulation, gives a 
better performance. The fixed shift demodulation is the scheme that we implemented 
in this thesis, which has the best performance among the three demodulation schemes 
according to the plot. Compared to the original demodulation scheme, the fixed shift 
gives a 0.6dB improvement for coherent modulation, and a 0.3dB improvement for 
differential modulation. 


coherent demod 




Figure 4-7: The error probability curves for three different demodulation algorithms 
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4.4 Summary 


In this chapter, we explored the problem of hireling the optimal bit-metric assignment 
for 8PSK modulated channel. 

First we showed that the marginal LLRs, are the bit metrics that maximize the 
generalized mutual information. This is also partially the reason that many BICM 
systems use it in the demodulator. However, this metric is not always optimal under 
different objective functions, e.g., the general cut-off rate. We derived the correspond¬ 
ing optimal bit metrics when max-log approximation holds, and they are simple linear 
functions that depend on the LLRs. 

We also developed an efficient algorithm that calculates the max-log approximated 
LLRs. The demodulation scheme based on this algorithm was implemented and tested 
on a physical chip. From the testing result, this scheme improves the system for 0.6dB 
in the coherent mode and for 0.3dB in the differential mode. 
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Chapter 5 


Conclusion and Future Work 


Finding the optimal demodulation schemes is a practical problem attracting much 
research interest. Various objective functions have been proposed as the optimiza¬ 
tion metric. In this thesis, we explored the problem from multiple aspects, which 
include maximizing different objective functions under different scenarios, and de¬ 
riving asymptotic solutions with low implementation cost that well approximate the 
numerical solutions. 

We started from the simplest case, where the modulation type of the signal is 2- 
PAM, and the channel output is quantized into finitely many regions. We developed 
approximate quantization schemes based on the small SNR asymptotic solutions of 
the optimal result. It gives negligible capacity loss in both the matched decoding and 
the mismatched decoding cases, while being simple to implement. We also studied 
the trade-off between the capacity loss and resolution of the quantizer. We showed 
that a very simple estimator of the relative loss of capacity can be derived in the 
matched decoding case using the high-rate limit approximation. 

Then we considered a more practical case where not only is the channel output 
quantized, but the decoding metrics are also constrained to given values. We showed 
that the small SNR approximation of the quantization thresholds still works in this 
case, which provides an approximation scheme that can be directly used on most 
modern communication devices. 

Finally we looked into the demodulation problem for 8PSK modulated channels. 
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We discussed the optimal bit-metric assignment for the GMI and the GCR. We also 
implemented a simple algorithm that decomposes the marginal LLRs based on the 
received symbol, which improves the performance of the chips where it was imple¬ 
mented. 

In addition to the topics we have discussed, there are still many related problems 
that remain open. 

1. In the case of matched decoding quantization, we have shown that the approx¬ 
imate quantization scheme derived from asymptotic solutions gives negligible 
capacity loss on binary input channels. However, solving the asymptotic solu¬ 
tions in the general cases, e.g. when 8PSK modulation is used, is less trivial. 
Thus, extending this approximation approach to different modulation types re¬ 
mains to be researched. 

2. In the analysis of the trade-off between the capacity loss and the quantizer reso¬ 
lution, we conjectured that the minimum relative loss of capacity is a decreasing 
function of the SNR for 2-PAM Gaussian channels. If this is true, we can derive 
a simple formula that estimates the worst case relative loss of the capacity in 
this scenario. 

3. Although the mismatched capacity is equal to the GMI for binary input chan¬ 
nels, it is not equal to the GMI lower bound, and its formula provided in [27] 
does not single-letterize in the general case. This makes it hard to maximize the 
actual mismatched capacity in many cases. Whether a simple formula exists for 
specific cases like 8PSK BICM channels, would be an interesting and valuable 
problem to research on. 
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Appendix A 


Proof of Theorem U 


Proof. First we apply the Gaussian channel condition to (2.4): 


z{y ) = argmax^y ^ P x e (s 2 v) ln(P x \z )} 


x 


(A.l) 


For Ji-PAM modulation, any possible channel input value x can be represented 
as: 


x = Xi = xo + id 


(A.2) 


Where i takes value from {0,1, 2 ,K — 1}. 


Then (A.l) can be further simplified: 


K -1 


(; V ) = argmax {^ P x e ^ +ld9y ln(P x \z)} 


z(y = 


(A.3) 


i =0 


We dehne variable <j> = e d9y . Then the quantization intervals of each quantized 
output z can be fully determined by a set of polynomial constraints: 


K -1 

^ ^ ( Pi,z ® 


(A.4) 


i=0 




for every z' z, with a,i tZ = P x e 2 1 ln(P x \z). 
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Since each polynomial constraint produces a set of disjoint intervals, and that 
the quantization intervals are the result of their intersection, the entire problem is 
reduced to a mathematical problem about counting the number of intervals. 

So we first introduce some definitions in order to simplify the following discussion. 

Definition 3. The boundaries of a union of disjoint intervals S = U^a*, bj\ are 
defined as U i{ai,bj}, denoted by B(S). 

Definition 4. The inner boundaries of a union of disjoint intervals S and defined 
as B(S) - {S max , S min }, denoted by B'(S). 

For simplicity, we denote the number of elements in an finite set B by \B\, and 
we denote the number of intervals in their union S to be | S\ 

We then prove the following lemma: 


Lemma 5. Given two arbitrary unions of disjoint intervals <Sj ,5V We have 


B'(S 1 DS 2 )\ < |£'(S , 1 )| + |£'(S , 2 )| 


(A.5) 


Proof. For an arbitrary set of disjoint intervals S, a necessary condition for a real 
number x to belong to B\S) is that x satisfies S mm < x < S max . 

Thus the following relation holds: 


B\Sins 2 ) = B(s 1 ns 2 ) 


-=>1 ,mm? ^ 2,max-) 


(A.6) 


It is easy to prove that 


B(S 1 n5 2 )CB(S 1 )uB(S 2 ). 


(A.7) 
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Then we have 


B'(S 1 n S 2 ) C ( B(S 0 - {5' 1 , max , 5 , i, min }) 

U (5(S , 2) — {‘S'2,moa:) ^2,min}) 
= j B / (^ 1 )U j B / (5 2 ) 


(A.8) 

(A.9) 


The following can thus be derived directly: 


|S , (5 , i n5 2 )| < | j B / (^ 1 )| + | j B / (5 2 )| 


(A.10) 


□ 


Now we can look at the intervals number counting problem we mentioned earlier. 


From (A.4), each single constraint generates a set of disjoint intervals. Since the 


constraints are polynomial on (j) with degrees no larger than K — 1, there are at most 
K — 1 roots for each constraint. We know that the number of inner boundaries has to 
be even, so the upper bound on the number of inner boundaries from one constraint 

is2L^J. 

Since there are IV — 1 constraints for each quantized output z. We ultilize Lemma 
[H] and conclude that |£?'(£(;?))| < 2(N — 1)|_A^J. 

It is straight forward to prove that |£> , (S')| = 2(|Sj — 1). So the upper bound 
on the number of disjoint intervals that map to the same quantized output in the 
optimal quantization scheme can be represented as: 


\S(z)\ < (iV — 1) [ 


K 


: J+1 


(A.ll) 


O 
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Appendix B 


The Second Order Expansion for 
Small SNR Matched Quantization 


We solve for the second order term of the quantization threshold 6 1; for the problem 
defined in 12.3.21 

First we expand the soft bit a to the second order: 

_ Q'(bo)x + Q" (bp)xbi + \Q'"(b Q )x 3 
Q(b o) + Q'(b 0 )bi + \Q"(bo)x 2 

Based the properties of b 0 and the Q function, the following equation holds: 


2b 0 Q(b 0 ) — —Q'(bo) 

Q"(b 0 ) = -boQ'ibo) 

Q'"(b 0 ) = (bl - l)Q\b 0 ) 

These equations can simplify the expression of a: 

1 - k,b,g 2 + ,'(/)“ - 1 )(/ 
“ 9 "25 + M 2 - |6o9 2 


(B-2) 

(B.3) 

(B.4) 


(B.5) 


Now we expand (2.12) to the second order: 
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(1 + 


(bog) 


a 


)(—a 2 — —) 
2 n 2 j 


+ (bo + i.9 2 )9(l + ^)(2o + 

o 3 


= 0 


(B-6) 


Plugging (B.5) into (B.6), we can solve for b 


h = ~-^b 0 

o 


(B.7) 
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Appendix C 


Large SNR Approximation for 
2PAM Mismatched Quantization 


by applying ZZZ = 0 to (3.13), we have 


-K 


y~^ Pz\x(z\l)g(z) 
l _|_ e otq{z) 


z=-l 


y~^ Pz\x{z\l)(l(z) 
Z_^ x + e aq W 


(C.l) 


z=1 


Using max-log approximation In JT x i ~ max* ln(:rt), C.l can be simplified to 


ln-Pz|x(-l|l)g(-l) = maxjln P z \x(z\l)q(z) - aq(z)} (C.2) 


We can approximate the transition probability of the quantized channel using 


lnP Z |x(z|l) ~ max 

y:Z(y)=z 


(■ y-9? 

2 


(C.3) 


Thus, 


In Pz\x{~ 111) 



(C.4) 
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For any 0 < z < K, the following inequality can be derived. 


In P z \x{z\l) < 


( 9 ~ min {b Zl g}) 2 
2 


(C.5) 


Then using (3.15), we have 


In P z \ x (z\l)q(z) 


otq(z) < -— + 0(\ng) 


(C.6) 


So C.2 can be simplified to 


9_ 

2 


\nP z \x(K\l) - aq(K) + 0(\ng) 
(.g ~ max{Fif i, g}) 2 


aq(K) + 0(ln g) 


The solution is 


(C.7) 

(C.8) 


« = 2 i + °( ln 9) 

< 


k 


— ML 

4 QK 


+ 0{ 1) 


(C.9) 
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Appendix D 


High-rate Limit for Mismatched 
Quantization with Integer Metrics 


Given a 2-PAM Gaussian channel with a quantize function Z(y), the loss of mis¬ 
matched capacity equals 


AG = min 

a 


fyJ2 P x\Y(x\y){ln 

X 


e xy 

Y, x >e x ' y 


— In 


e <xq(x,z) 

Ex' 


)dy 


(D.l) 


When the number of quantization region is large, the optimal relative decoding 
metric q(z) should match with the actual log-likelihood ratio, so that the loss of 
capacity can approach zero, i.e., 


aq(z) ss 2gy 


(D.2) 


with probability of 1. 


Thus, we can Taylor expand (D.l) and derive the capacity loss in the high-rate 
limit. 


AG ~ min 

a 


f y (ad - 2gy) 2 
2 (e3v + e-9yy y 


(D.3) 
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The approximate loss of capacity, defined by (D.3), can be calculated by dividing 
the integral into 2 parts: \2gy\ < aK and \2gy\ > aK. For convenience, we define 
this boundary to be t/ m = 


The first part of the integral can be treated by assuming the length of the quan¬ 
tization intervals approaches to 0 when the number of quantization regions N is 
large. Thus the coefficients of the “square error” function is almost constant within 
a interval. 


f fy{ad-2gy) 2 

I\2 gy \<aK 2(e 9y + e~ 9y ) 2 V ' 


f f y Vai(2gy\Z = Z(y)) 

l\2 9 y\<aK 2{e 9y + e~ 9y ) 2 y 

r+°° f \2 


(D.4) 

(D.5) 


To simplify the above equation, we substitute the coefficients that are independent 
of the quantizer parameters a and N. 


A 



2f y g 2 

3{e" + e~ 9y ) 2 


dy 


(D.6) 


Then the first part of the integral can be simply expressed as Ay^/N 2 . 

Before calculating the second part of the integral, we derive the following lemma. 


Lemma 6. For any x > 0, the following inequality hold. 


2 — e ( X+ I)( x 2 + 3x + f + °° (x+yf 


(x+l ) 3 


e 2 < / e 2 


y 2 dy < —^e~ 


x J 


Proof. First we derive the upper bound 


"+°° _(x+yP 2 

e 2 y dy < 


r»+QO 2 

-^- X y ni 2 


y dy 


'0 


= - P 2 

X s 


(D.7) 


(D.8) 

(D.9) 
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Then we derive the lower bound 


r ‘ + °° (x+y ) 2 2 f 1 _^_ (x+ i )y 2 7 

e 2 y z dy > e 2 (x+2>y y dy 


2 — e ( X +I)( x 2 + 3x + *£ 


(z+ \) 3 


-e 2 


(D.10) 

(D.ll) 

□ 


From Lemma [6] we obtained a convenient approximation 


f + °° (*+y) 2 g , 2 _x 2 

/ e 2 y dy ^ —e 2 

/ 0 ar 


which simplifies the second part of the integration of the capacity loss. 


r f y (ad-2gy ) 2 

' \2gy\>aK 2(e ra + e~ 9v ) 2 


dy 


4 g 2 (g+y) 


'y>y m V / 27T 

8 g 2 


e {y-y m ) 2 dy 

(g+l/m)^ 


VMfl + i/m) 3 


(D. 12) 


(D.13) 

(D.14) 


Similarly, we make the following substitution for the coefficient of the second 
integral. 


B = 


8 g 2 


(D.15) 


Combining (D.6) and (D.14), we obtain a simple formula that approximates the 
capacity loss. 


AC 


• r A dm 

111111 1 ACT 

J/m iv 


+ 


B 


(.9 + Vm) 3 


( 9 +Vrr 

2 


(D.16) 


When y m ^> 1, with some simple analysis, the loss of capacity is minimized at 


2/ m = V4 In N + 0(1) 


(D.17) 
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and the capacity loss equals 


4A In N Chi N 


N 2 


N 2 


(D. 18) 


So far, we have derived the minimum mismatched capacity loss for 2-PAM Gaus¬ 


sian Channels in the high-rate limit. In terms of using (D.18) as a estimator of the 
capacity loss, it requires Vin N 1, which in practice is not true. So the approxi¬ 
mate formula of capacity loss in the mismatched case can not be derived from this 
approach. 







Appendix E 


Optimal Demodulation Maximizing 
GCR for 8PSK Gaussian Channel 


Finding the optimal bit metrics that maximizes the GCR is essentially solving equal- 


tion (4.13). Although finding an exact formula of the solution is generally difficult, we 


can instead derive an approximate result when max-log approximation can be used, 
i.e., solving the below equation instead. 


V' ' e aq(x',y) 

(?i , 92 , ®0 = argmin max P X y {x, y) , , — 

91,92,93 1 e a ^ x ’ y > 


(E.l) 


Specifically, for 8PSK Gaussian channel with the labeling of constellation points 
specified in Fig. |4-1 the above equation is equivalent to 


91,92,93 


(qi, q 2 , q 3 ) = argmin JJ(1 + e~ aqi )(e 9yl + e 9 ^ 1+aqs + e w-e**+«(M+®) 

■ ~ j 

i ^ 7r • a 5tt 

-\- e 9V' e T +“92 _|_ g9J/-e* 7r +“(9i +92) g gy-e T +a(<ji +92+93) 

_|_g9J/' e “ -r«wi-r93; gy 


o 37T -77r 

^+“(91+93) 1 „92/-e t +“9! n 


(E.2) 


To simplify our discussion, we divide the channel output space into 8 symmetric 


regions (see Fig. E-l) based on the complex phase of the channel output y. We shall 


only derive the solution to (E.2) when the channel output phase belongs to 


7T 7T1 

’ 8 ’ 8 J 5 


and the rest part of the solution can be easily obtained due to the symmetry. 
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Q 




Figure E-l: The 8-region devision of the channel output space for 8PSK demodula¬ 
tion. 


When the channel output belongs to the region specified above, we have the 
following inequalities 


y ■ 1 > y ■ e ±l 4 > y ■ e ±l 2 > y ■ e ±l ^ > y ■ e m 


(E.3) 


Then the target function defined in (E.2) is lower bounded by e y ' 1 , and this expo¬ 
nent can be achieved when e aqi 1 and the following inequalities hold. 


V ■ 1 > ye^ + fa 
y- 1> y-e*% + f(g 2 + g 3 ) 
y ■ 1 > ye iS r + Zq 2 

< y ■ 1 > y ■ e in + |(gi + q 2 ) ( E - 4 ) 

y -1 > ye**? + f(gi + g 2 + ga) 
y- 1 > ye i? r + |(gi + g 3 ) 

■ 1 > y ■ e iJ r + 1 9l 


Applying (E.3) to the above inequalities, we can derive a simplified version of the 
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constraints on the optimal bit metrics. 


Qi > 0 

y ■ i > v ■ e** + f 93 
y • i > ye^ + f(g 2 + g 3 ) 
y • i > y ■ | gi 


(E.5) 


Now we start solving for the optimal values of the decoding metrics. Taking the 


partial derivative of the objective function in (E.2) with respect to the bit metrics, 
we obtain the equations that restrict its solution. 


2aqi 


2 aq 2 


2 aq 3 


log 


log 


log 


e 9y t _|_ e gy-e^+aq3 _|_ e gy-e% +a(q 2 +qa.) _|_ e gy-e^T +aq 2 

,577 -37T -777 

e gy e m +aq 2 _|_ e gy-e T+ Q ( g2 +g 3 ) _|_ e gy-e +aq 3 _|_ e gy-e T 

j 77 j 377 j 7 77 

e 92/'l _|_ g3J/' e 7 +«93 _)_ e 3?/-e 2 +a(gi+<?3) _|_ g3?/-e T+aqi 

e gy-e l ^+aq 3 e gy e _)_ e gy-e i7r +aqi _)_ e gy-e’V+ a (gi+g 3 ) 

, 377 . -777 

gOT'l _)_ e ff2/-e T +a<22 _j_ e 32/-e I7r +a(gi+ ( }2) _|_ ^+aqi 

A 77 , 77 -577 -377 

e S2/-e ^ _|_ e9 y. e ^+a<?2 _)_ g3V e T +a(gi+<?2) _|_ gS2/e "2" +a<ji 


(E.6) 

(E-7) 

(E-8) 


Utilizing in inequalities in (E.5) and the max-log approximation, the above equa¬ 
tions can be simplified into 


2aqi = gy • 1 - ma x{gy ■ e m + aq 2 , gy ■ e 1 ^ + aq 3 , gy ■ e ‘t} 
2aq 2 = gy ■ 1 - gy ■ e^ - aq 3 
2 aq 3 = gy- 1 - ma x{gy ■ e*U • e** + ag 2 } 


(E.9) 

(E.10) 

(E.ll) 


Solving the above equations, we derive the formulas for the optimal solution, which 
can also be expressed using their corresponding log-likelihood ratios. 
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(E.12) 


Qi = 

q 2 = 

Q3 = 


gy( l-e^) LLRi 

2a 2a 

gy ■ (1 + — 2e*^) LLR 2 — \LLR 3 

4a 2a 

gy • (1 ~ e»5) = LLR 3 
2 a 2a 


(E. 13) 
(E.14) 


Since scaling the quantization metrics does not affect the decoding result, the 
parameter a can be an arbitrary positive value. 
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